Tootfinder

Of geek interest: UUIDs have been around for a long time and they work well and interoperate fine, but the specification space is a mess. So the IETF just shipped RFC9562; it has a fine consideration of the history and the trade-offs involved in all the different flavors of UUIDs, and is well-written and I think, as of now, the place to start looking if something UUID-flavored is puzzling you. Strong work!

Analysis of Distributed Algorithms for Big-data
Rajendra Purohit, K R Chowdhary, S D Purohit
https://arxiv.org/abs/2404.06461 https://

Analysis of Distributed Algorithms for Big-data
The parallel and distributed processing are becoming de facto industry standard, and a large part of the current research is targeted on how to make computing scalable and distributed, dynamically, without allocating the resources on permanent basis. The present article focuses on the study and performance of distributed and parallel algorithms their file systems, to achieve scalability at local level (OpenMP platform), and at global level where computing and file systems are distributed. Vario…

Of geek interest: UUIDs have been around for a long time and they work well and interoperate fine, but the specification space is a mess. So the IETF just shipped RFC9562; it has a fine consideration of the history and the trade-offs involved in all the different flavors of UUIDs, and is well-written and I think, as of now, the place to start looking if something UUID-flavored is puzzling you. Strong work!

Software and computing for Run 3 of the ATLAS experiment at the LHC
ATLAS Collaboration
https://arxiv.org/abs/2404.06335 https://arxi…

Software and computing for Run 3 of the ATLAS experiment at the LHC
The ATLAS experiment has developed extensive software and distributed computing systems for Run 3 of the LHC. These systems are described in detail, including software infrastructure and workflows, distributed data and workload management, database infrastructure, and validation. The use of these systems to prepare the data for physics analysis and assess its quality are described, along with the software tools used for data analysis itself. An outlook for the development of these projects towa…

Optimal Allocation of Tasks and Price of Anarchy of Distributed Optimization in Networked Computing Facilities
Vincenzo Mancuso, Paolo Castagno, Leonardo Badia, Matteo Sereno, Marco Ajmone Marsan
https://arxiv.org/abs/2404.05543

Optimal Allocation of Tasks and Price of Anarchy of Distributed Optimization in Networked Computing Facilities
The allocation of computing tasks for networked distributed services poses a question to service providers on whether centralized allocation management be worth its cost. Existing analytical models were conceived for users accessing computing resources with practically indistinguishable (hence irrelevant for the allocation decision) delays, which is typical of services located in the same distant data center. However, with the rise of the edge-cloud continuum, a simple analysis of the sojourn t…

RIS-empowered Topology Control for Distributed Learning in Urban Air Mobility
Kai Xiong, Rui Wang, Supeng Leng, Wenyang Che, Chongwen Huang, Chau Yuen
https://arxiv.org/abs/2403.05133

RIS-empowered Topology Control for Distributed Learning in Urban Air Mobility
Urban Air Mobility (UAM) expands vehicles from the ground to the near-ground space, envisioned as a revolution for transportation systems. Comprehensive scene perception is the foundation for autonomous aerial driving. However, UAM encounters the intelligent perception challenge: high perception learning requirements conflict with the limited sensors and computing chips of flying cars. To overcome the challenge, federated learning (FL) and other collaborative learning have been proposed to enab…

Communication-Efficient Large-Scale Distributed Deep Learning: A Comprehensive Survey
Feng Liang, Zhen Zhang, Haifeng Lu, Victor C. M. Leung, Yanyi Guo, Xiping Hu
https://arxiv.org/abs/2404.06114

Communication-Efficient Large-Scale Distributed Deep Learning: A Comprehensive Survey
With the rapid growth in the volume of data sets, models, and devices in the domain of deep learning, there is increasing attention on large-scale distributed deep learning. In contrast to traditional distributed deep learning, the large-scale scenario poses new challenges that include fault tolerance, scalability of algorithms and infrastructures, and heterogeneity in data sets, models, and resources. Due to intensive synchronization of models and sharing of data across GPUs and computing node…

The Voronoi Diagram of Weakly Smooth Planar Point Sets in $O(\log n)$ Deterministic Rounds on the Congested Clique
Jesper Jansson, Christos Levcopoulos, Andrzej Lingas
https://arxiv.org/abs/2404.06068

The Voronoi Diagram of Weakly Smooth Planar Point Sets in $O(\log n)$ Deterministic Rounds on the Congested Clique
We study the problem of computing the Voronoi diagram of a set of $n^2$ points with $O(\log n)$-bit coordinates in the Euclidean plane in a substantially sublinear in $n$ number of rounds in the congested clique model with $n$ nodes. Recently, Jansson et al. have shown that if the points are uniformly at random distributed in a unit square then their Voronoi diagram within the square can be computed in $O(1)$ rounds with high probability (w.h.p.). We show that if a very weak smoothness conditio…

Distributed Adaptive Spatial Filtering with Inexact Local Solvers
Charles Hovine, Alexander Bertrand
https://arxiv.org/abs/2405.03277 https://

Distributed Adaptive Spatial Filtering with Inexact Local Solvers
The Distributed Adaptive Signal Fusion (DASF) framework is a meta-algorithm for computing data-driven spatial filters in a distributed sensing platform with limited bandwidth and computational resources, such as a wireless sensor network. The convergence and optimality of the DASF algorithm has been extensively studied under the assumption that an exact, but possibly impractical solver for the local optimization problem at each updating node is available. In this work, we provide convergence an…

Photonic next-generation reservoir computer based on distributed feedback in optical fiber
Nicholas Cox, Joseph Murray, Joseph Hart, Brandon Redding
https://arxiv.org/abs/2404.07116

Photonic next-generation reservoir computer based on distributed feedback in optical fiber
Reservoir computing (RC) is a machine learning paradigm that excels at dynamical systems analysis. Photonic RCs, which perform implicit computation through optical interactions, have attracted increasing attention due to their potential for low latency predictions. However, most existing photonic RCs rely on a nonlinear physical cavity to implement system memory, limiting control over the memory structure and requiring long warm-up times to eliminate transients. In this work, we resolve these i…

Atomicity in Distributed Quantum Computing
Zhicheng Zhang, Mingsheng Ying
https://arxiv.org/abs/2404.18592 https://arxiv.org/pdf/2404…

Atomicity in Distributed Quantum Computing
Atomicity is a ubiquitous assumption in distributed computing, under which actions are indivisible and appear sequential. In classical computing, this assumption has several theoretical and practical guarantees. In quantum computing, although atomicity is still commonly assumed, it has not been seriously studied, and a rigorous basis for it is missing. Classical results on atomicity do not directly carry over to distributed quantum computing, due to new challenges caused by quantum entanglement…

This https://arxiv.org/abs/2403.01296 has been replaced.
link: https://scholar.google.com/scholar?q=a

Rate-limited Shuffling for Distributed Computing
This paper studies the shuffling phase in a distributed computing model with rate-limited links between nodes. Each node is connected to all other nodes via a noiseless broadcast link with a finite capacity. For this network, the shuffling phase is described as a distributed index-coding problem to extend an outer bound for the latter to the distributed computing problem. An inner bound on the capacity region is also established by using the distributed composite-coding scheme introduced for th…

[2024-05-10 Fri (UTC), 6 new articles found for cs.DC Distributed, Parallel, and Cluster Computing]

Snake Learning: A Communication- and Computation-Efficient Distributed Learning Framework for 6G
Xiaoxue Yu, Xingfu Yi, Rongpeng Li, Fei Wang, Chenghui Peng, Zhifeng Zhao, Honggang Zhang
https://arxiv.org/abs/2405.03372

Snake Learning: A Communication- and Computation-Efficient Distributed Learning Framework for 6G
In the evolution towards 6G, integrating Artificial Intelligence (AI) with advanced network infrastructure emerges as a pivotal strategy for enhancing network intelligence and resource utilization. Existing distributed learning frameworks like Federated Learning and Split Learning often struggle with significant challenges in dynamic network environments including high synchronization demands, costly communication overheads, severe computing resource consumption, and data heterogeneity across n…

[2024-04-10 Wed (UTC), 6 new articles found for cs.DC Distributed, Parallel, and Cluster Computing]

Even CERN has to queue for GPUs. Here's how they optimise what they have.
'There's a tendency to say that all ML workloads need a GPU, but for inference you probably don't need them' (free reg)
https://www.computing.co.uk/interview/4197

Even CERN has to queue for GPUs. Here's how they optimise what they have
CERN, the European Organisation for Nuclear Research, has long had some of the biggest of big data needs. Its engineers historically built their own IT systems including for distributed and high performing computing (HPC), and sometimes still do. But the culture has largely shifted to engaging with open source communities with similar requirements, including adopting OpenStack for the computing needs of the Large Hadron Collider.

Perfect Subset Privacy in Polynomial Computation
Zirui (Ken), Deng, Vinayak Ramkumar, Netanel Raviv
https://arxiv.org/abs/2405.05567 https://

Perfect Subset Privacy in Polynomial Computation
Delegating large-scale computations to service providers is a common practice which raises privacy concerns. This paper studies information-theoretic privacy-preserving delegation of data to a service provider, who may further delegate the computation to auxiliary worker nodes, in order to compute a polynomial over that data at a later point in time. We study techniques which are compatible with robust management of distributed computation systems, an area known as coded computing. Privacy in c…

[2024-04-11 Thu (UTC), 3 new articles found for cs.DC Distributed, Parallel, and Cluster Computing]

Diese Studie wurde offenbar mit #ChatGPT geschrieben und die Autoren haben es wohl versäumt, Korrektur zu lesen.
Finden den Fehler: 🤔 😉
"Scalability and Distributed Computing in NET for Large-Scale AI Workloads"
#Science

Age-of-Information-Aware Distributed Task Offloading and Resource Allocation in Mobile Edge Computing Networks
Minwoo Kim, Jonggyu Jang, Youngchol Choi, Hyun Jong Yang
https://arxiv.org/abs/2404.03154

Age-of-Information-Aware Distributed Task Offloading and Resource Allocation in Mobile Edge Computing Networks
The growth in artificial intelligence (AI) technology has attracted substantial interests in age-of-information (AoI)-aware task offloading of mobile edge computing (MEC)-namely, minimizing service latency. Additionally, the use of MEC systems poses an additional problem arising from limited battery resources of MDs. This paper tackles the pressing challenge of AoI-aware distributed task offloading optimization, where user association (UA), resource allocation (RA), full-task offloading, and ba…

Content-Oblivious Leader Election on Rings
Fabian Frei, Ran Gelles, Ahmed Ghazy, Alexandre Nolin
https://arxiv.org/abs/2405.03646 https://

Content-Oblivious Leader Election on Rings
In content-oblivious computation, n nodes wish to compute a given task over an asynchronous network that suffers from an extremely harsh type of noise, which corrupts the content of all messages across all channels. In a recent work, Censor-Hillel, Cohen, Gelles, and Sela (Distributed Computing, 2023) showed how to perform arbitrary computations in a content-oblivious way in 2-edge connected networks but only if the network has a distinguished node (called root) to initiate the computation. O…

Enhancing Trust and Privacy in Distributed Networks: A Comprehensive Survey on Blockchain-based Federated Learning
Ji Liu, Chunlu Chen, Yu Li, Lin Sun, Yulun Song, Jingbo Zhou, Bo Jing, Dejing Dou
https://arxiv.org/abs/2403.19178

Enhancing Trust and Privacy in Distributed Networks: A Comprehensive Survey on Blockchain-based Federated Learning
While centralized servers pose a risk of being a single point of failure, decentralized approaches like blockchain offer a compelling solution by implementing a consensus mechanism among multiple entities. Merging distributed computing with cryptographic techniques, decentralized technologies introduce a novel computing paradigm. Blockchain ensures secure, transparent, and tamper-proof data management by validating and recording transactions via consensus across network nodes. Federated Learnin…

[2024-04-11 Thu (UTC), 3 new articles found for cs.DC Distributed, Parallel, and Cluster Computing]

This https://arxiv.org/abs/2308.00144 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csDC_…

Logical Synchrony and the bittide Mechanism
We introduce logical synchrony, a framework that allows distributed computing to be coordinated as tightly as in synchronous systems without the distribution of a global clock or any reference to universal time. We develop a model of events called a logical synchrony network, in which nodes correspond to processors and every node has an associated local clock which generates the events. We construct a measure of logical latency and develop its properties. A further model, called a multiclock ne…

This https://arxiv.org/abs/2308.00144 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csDC_…

Logical Synchrony and the bittide Mechanism
We introduce logical synchrony, a framework that allows distributed computing to be coordinated as tightly as in synchronous systems without the distribution of a global clock or any reference to universal time. We develop a model of events called a logical synchrony network, in which nodes correspond to processors and every node has an associated local clock which generates the events. We construct a measure of logical latency and develop its properties. A further model, called a multiclock ne…

Fast Computation of Robust Dynamic Operating Envelopes Based on Non-convex OPF for Unbalanced Distribution Networks
Bin Liu, Julio H. Braslavsky
https://arxiv.org/abs/2404.03355 <…

Fast Computation of Robust Dynamic Operating Envelopes Based on Non-convex OPF for Unbalanced Distribution Networks
Robust dynamic operating envelopes (RDOEs) solve the problem of secure allocation of latent network capacity to flexible distributed energy resources (DER) in unbalanced distribution networks. As the computational complexity of RDOEs is much higher than that of DOEs, which disregard uncertainties in network parameters and DER capacity utilisation, existing approaches to computing RDOEs have relied on linearised unbalanced three-phase optimal power flow (UTOPF) models to numerate the network fea…

This https://arxiv.org/abs/2302.09597 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_ees…

Solving Differential-Algebraic Equations in Power System Dynamic Analysis with Quantum Computing
Power system dynamics are generally modeled by high dimensional nonlinear differential-algebraic equations (DAEs) given a large number of components forming the network. These DAEs' complexity can grow exponentially due to the increasing penetration of distributed energy resources, whereas their computation time becomes sensitive due to the increasing interconnection of the power grid with other energy systems. This paper demonstrates the use of quantum computing algorithms to solve DAEs for po…

[2024-03-11 Mon (UTC), 2 new articles found for cs.DC Distributed, Parallel, and Cluster Computing]

Distributed Radiance Fields for Edge Video Compression and Metaverse Integration in Autonomous Driving
Eugen \v{S}lapak, Mat\'u\v{s} Dopiriak, Mohammad Abdullah Al Faruque, Juraj Gazda, Marco Levorato
https://arxiv.org/abs/2402.14642

Distributed Radiance Fields for Edge Video Compression and Metaverse Integration in Autonomous Driving
The metaverse is a virtual space that combines physical and digital elements, creating immersive and connected digital worlds. For autonomous mobility, it enables new possibilities with edge computing and digital twins (DTs) that offer virtual prototyping, prediction, and more. DTs can be created with 3D scene reconstruction methods that capture the real world's geometry, appearance, and dynamics. However, sending data for real-time DT updates in the metaverse, such as camera images and videos …

A Selective Review on Statistical Methods for Massive Data Computation: Distributed Computing, Subsampling, and Minibatch Techniques
Xuetong Li, Yuan Gao, Hong Chang, Danyang Huang, Yingying Ma, Rui Pan, Haobo Qi, Feifei Wang, Shuyuan Wu, Ke Xu, Jing Zhou, Xuening Zhu, Yingqiu Zhu, Hansheng Wang
https://arxiv.org/abs/2403.11163<…

A Selective Review on Statistical Methods for Massive Data Computation: Distributed Computing, Subsampling, and Minibatch Techniques
This paper presents a selective review of statistical computation methods for massive data analysis. A huge amount of statistical methods for massive data computation have been rapidly developed in the past decades. In this work, we focus on three categories of statistical computation methods: (1) distributed computing, (2) subsampling methods, and (3) minibatch gradient techniques. The first class of literature is about distributed computing and focuses on the situation, where the dataset size…

🐧 Join MacKenzie Adam for a journey into the future of cloud computing with "Cloud Native WebAssembly." Explore how #WebAssembly is reshaping serverless and distributed computing. https://www.

This https://arxiv.org/abs/2111.03683 has been replaced.
link: https://scholar.google.com/scholar?q=a

On Homomorphism Graphs
We introduce a new type of examples of bounded degree acyclic Borel graphs and study their combinatorial properties in the context of descriptive combinatorics, using a generalization of the determinacy method of Marks. The motivation for the construction comes from the adaptation of this method to the LOCAL model of distributed computing. Our approach unifies the previous results in the area, as well as produces new ones. In particular, we show that for $Δ>2$ it is impossible to give a simple…

NotNets: Accelerating Microservices by Bypassing the Network
Peter Alvaro, Matthew Adiletta, Adrian Cockroft, Frank Hady, Ramesh Illikkal, Esteban Ramos, James Tsai, Robert Soul\'e
https://arxiv.org/abs/2404.06581

NotNets: Accelerating Microservices by Bypassing the Network
Remote procedure calls are the workhorse of distributed systems. However, as software engineering trends, such as micro-services and serverless computing, push applications towards ever finer-grained decompositions, the overhead of RPC-based communication is becoming too great to bear. In this paper, we argue that point solutions that attempt to optimize one aspect of RPC logic are unlikely to mitigate these ballooning communication costs. Rather, we need a dramatic reappraisal of how we provid…

NotNets: Accelerating Microservices by Bypassing the Network
Peter Alvaro, Matthew Adiletta, Adrian Cockroft, Frank Hady, Ramesh Illikkal, Esteban Ramos, James Tsai, Robert Soul\'e
https://arxiv.org/abs/2404.06581

NotNets: Accelerating Microservices by Bypassing the Network
Remote procedure calls are the workhorse of distributed systems. However, as software engineering trends, such as micro-services and serverless computing, push applications towards ever finer-grained decompositions, the overhead of RPC-based communication is becoming too great to bear. In this paper, we argue that point solutions that attempt to optimize one aspect of RPC logic are unlikely to mitigate these ballooning communication costs. Rather, we need a dramatic reappraisal of how we provid…

EdgeLeakage: Membership Information Leakage in Distributed Edge Intelligence Systems
Kongyang Chen, Yi Lin, Hui Luo, Bing Mi, Yatie Xiao, Chao Ma, Jorge S\'a Silva
https://arxiv.org/abs/2404.16851

EdgeLeakage: Membership Information Leakage in Distributed Edge Intelligence Systems
In contemporary edge computing systems, decentralized edge nodes aggregate unprocessed data and facilitate data analytics to uphold low transmission latency and real-time data processing capabilities. Recently, these edge nodes have evolved to facilitate the implementation of distributed machine learning models, utilizing their computational resources to enable intelligent decision-making, thereby giving rise to an emerging domain referred to as edge intelligence. However, within the realm of e…

This https://arxiv.org/abs/2402.04377 has been replaced.
link: https://scholar.google.com/scholar?q=a

NeRCC: Nested-Regression Coded Computing for Resilient Distributed Prediction Serving Systems
Resilience against stragglers is a critical element of prediction serving systems, tasked with executing inferences on input data for a pre-trained machine-learning model. In this paper, we propose NeRCC, as a general straggler-resistant framework for approximate coded computing. NeRCC includes three layers: (1) encoding regression and sampling, which generates coded data points, as a combination of original data points, (2) computing, in which a cluster of workers run inference on the coded da…

Collaborative Optimization of Wireless Communication and Computing Resource Allocation based on Multi-Agent Federated Weighting Deep Reinforcement Learning
Junjie Wu, Xuming Fang
https://arxiv.org/abs/2404.01638

Collaborative Optimization of Wireless Communication and Computing Resource Allocation based on Multi-Agent Federated Weighting Deep Reinforcement Learning
As artificial intelligence (AI)-enabled wireless communication systems continue their evolution, distributed learning has gained widespread attention for its ability to offer enhanced data privacy protection, improved resource utilization, and enhanced fault tolerance within wireless communication applications. Federated learning further enhances the ability of resource coordination and model generalization across nodes based on the above foundation, enabling the realization of an AI-driven com…

Secure Coded Distributed Computing
Shanuja Sasi, Onur G\"unl\"u
https://arxiv.org/abs/2404.16431 https://arxiv.org/pdf/2404…

Secure Coded Distributed Computing
In this paper, we consider two critical aspects of security in the \textit{distributed computing (DC)} model: \textit{secure data shuffling} and \textit{secure coded computing}. It is imperative that any external entity overhearing the transmissions does not gain any information about the \textit{intermediate values (IVs)} exchanged during the shuffling phase of the DC model. Our approach ensures IV confidentiality during data shuffling. Moreover, each node in the system must be able to recover…

[2024-05-09 Thu (UTC), 7 new articles found for cs.DC Distributed, Parallel, and Cluster Computing]

This https://arxiv.org/abs/2404.16431 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csIT_…

Secure Coded Distributed Computing
In this paper, we consider two critical aspects of security in the \textit{distributed computing (DC)} model: \textit{secure data shuffling} and \textit{secure coded computing}. It is imperative that any external entity overhearing the transmissions does not gain any information about the \textit{intermediate values (IVs)} exchanged during the shuffling phase of the DC model. Our approach ensures IV confidentiality during data shuffling. Moreover, each node in the system must be able to recover…

This https://arxiv.org/abs/2207.04939 has been replaced.
link: https://scholar.google.com/scholar?q=a

Resource analysis for quantum-aided Byzantine agreement with the four-qubit singlet state
In distributed computing, a Byzantine fault is a condition where a component behaves inconsistently, showing different symptoms to different components of the system. Consensus among the correct components can be reached by appropriately crafted communication protocols even in the presence of byzantine faults. Quantum-aided protocols built upon distributed entangled quantum states are worth considering, as they are more resilient than traditional ones. Based on earlier ideas, here we establish …

Tessellated Distributed Computing
Ali Khalesi, Petros Elia
https://arxiv.org/abs/2404.14203 https://arxiv.org/pdf/2404.14203

Tessellated Distributed Computing
The work considers the $N$-server distributed computing scenario with $K$ users requesting functions that are linearly-decomposable over an arbitrary basis of $L$ real (potentially non-linear) subfunctions. In our problem, the aim is for each user to receive their function outputs, allowing for reduced reconstruction error (distortion) $ε$, reduced computing cost ($γ$; the fraction of subfunctions each server must compute), and reduced communication cost ($δ$; the fraction of users each serv…

Urgent Edge Computing
Patrizio Dazzi, Luca Ferrucci, Marco Danelutto, Konstantinos Tserpes, Antonis Makris, Theodoros Theodoropoulos, Jacopo Massa, Emanuele Carlini, Matteo Mordacchini
https://arxiv.org/abs/2404.13411

Urgent Edge Computing
This position paper introduces Urgent Edge Computing (UEC) as a paradigm shift addressing the evolving demands of time-sensitive applications in distributed edge environments, in time-critical scenarios. With a focus on ultra-low latency, availability, resource management, decentralization, self-organization, and robust security, UEC aims to facilitate operations in critical scenarios such as disaster response, environmental monitoring, and smart city management. This paper outlines and discuss…

Scheduling of Distributed Applications on the Computing Continuum: A Survey
Narges Mehran, Dragi Kimovski, Hermann Hellwagner, Dumitru Roman, Ahmet Soylu, Radu Prodan
https://arxiv.org/abs/2405.00005

Scheduling of Distributed Applications on the Computing Continuum: A Survey
The demand for distributed applications has significantly increased over the past decade, with improvements in machine learning techniques fueling this growth. These applications predominantly utilize Cloud data centers for high-performance computing and Fog and Edge devices for low-latency communication for small-size machine learning model training and inference. The challenge of executing applications with different requirements on heterogeneous devices requires effective methods for solving…

Mitigating Data Sharing in Public Cloud using Blockchain
Pratik Patil, Prerna Tulsiani, Dr. Sunil Mane
https://arxiv.org/abs/2404.16872 https://

Mitigating Data Sharing in Public Cloud using Blockchain
Public Cloud Computing has become a fundamental part of modern IT infrastructure as its adoption has transformed the way businesses operate. However, cloud security concerns introduce new risks and challenges related to data protection, sharing, and access control. A synergistic integration of blockchain with the cloud holds immense potential. Blockchain's distributed ledger ensures transparency, immutability, and efficiency as it reduces the reliance on centralized authorities. Motivated by th…

This https://arxiv.org/abs/2312.04141 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csIT_…

Distributed Approximate Computing with Constant Locality
Consider a distributed coding for computing problem with constant decoding locality, i.e., with a vanishing error probability, any single sample of the function can be approximately recovered by probing only constant number of compressed bits. We establish an achievable rate region by designing an efficient layered coding scheme, where the coding rate is reduced by introducing auxiliary random variables and local decoding is achieved by exploiting the expander graph code. Then we show the rate …

This https://arxiv.org/abs/2403.00331 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csDC_…

WindGP: Efficient Graph Partitioning on Heterogenous Machines
Graph Partitioning is widely used in many real-world applications such as fraud detection and social network analysis, in order to enable the distributed graph computing on large graphs. However, existing works fail to balance the computation cost and communication cost on machines with different power (including computing capability, network bandwidth and memory size), as they only consider replication factor and neglect the difference of machines in realistic data centers. In this paper, we p…

[2024-03-08 Fri (UTC), 5 new articles found for cs.DC Distributed, Parallel, and Cluster Computing]

[2024-03-08 Fri (UTC), 5 new articles found for cs.DC Distributed, Parallel, and Cluster Computing]

Multiple Access in the Era of Distributed Computing and Edge Intelligence
Nikos G. Evgenidis, Nikos A. Mitsiou, Vasiliki I. Koutsioumpa, Sotiris A. Tegos, Panagiotis D. Diamantoulakis, George K. Karagiannidis
https://arxiv.org/abs/2403.07903

Multiple Access in the Era of Distributed Computing and Edge Intelligence
This paper focuses on the latest research and innovations in fundamental next-generation multiple access (NGMA) techniques and the coexistence with other key technologies for the sixth generation (6G) of wireless networks. In more detail, we first examine multi-access edge computing (MEC), which is critical to meeting the growing demand for data processing and computational capacity at the edge of the network, as well as network slicing. We then explore over-the-air (OTA) computing, which is co…

Enhancing IoT Security: A Novel Feature Engineering Approach for ML-Based Intrusion Detection Systems
Afsaneh Mahanipour, Hana Khamfroush
https://arxiv.org/abs/2404.19114 https://arxiv.org/pdf/2404.19114
arXiv:2404.19114v1 Announce Type: new
Abstract: The integration of Internet of Things (IoT) applications in our daily lives has led to a surge in data traffic, posing significant security challenges. IoT applications using cloud and edge computing are at higher risk of cyberattacks because of the expanded attack surface from distributed edge and cloud services, the vulnerability of IoT devices, and challenges in managing security across interconnected systems leading to oversights. This led to the rise of ML-based solutions for intrusion detection systems (IDSs), which have proven effective in enhancing network security and defending against diverse threats. However, ML-based IDS in IoT systems encounters challenges, particularly from noisy, redundant, and irrelevant features in varied IoT datasets, potentially impacting its performance. Therefore, reducing such features becomes crucial to enhance system performance and minimize computational costs. This paper focuses on improving the effectiveness of ML-based IDS at the edge level by introducing a novel method to find a balanced trade-off between cost and accuracy through the creation of informative features in a two-tier edge-user IoT environment. A hybrid Binary Quantum-inspired Artificial Bee Colony and Genetic Programming algorithm is utilized for this purpose. Three IoT intrusion detection datasets, namely NSL-KDD, UNSW-NB15, and BoT-IoT, are used for the evaluation of the proposed approach.

Distributed OpenMP Offloading of OpenMC on Intel GPU MAX Accelerators
Yehonatan Fridman, Guy Tamir, Uri Steinitz, Gal Oren
https://arxiv.org/abs/2403.02735

Distributed OpenMP Offloading of OpenMC on Intel GPU MAX Accelerators
Monte Carlo (MC) simulations play a pivotal role in diverse scientific and engineering domains, with applications ranging from nuclear physics to materials science. Harnessing the computational power of high-performance computing (HPC) systems, especially Graphics Processing Units (GPUs), has become essential for accelerating MC simulations. This paper focuses on the adaptation and optimization of the OpenMC neutron and photon transport Monte Carlo code for Intel GPUs, specifically the Intel Da…

CARISMA: CAR-Integrated Service Mesh Architecture
Kevin Klein, Pascal Hirmer, Steffen Becker
https://arxiv.org/abs/2403.04378 https://

CARISMA: CAR-Integrated Service Mesh Architecture
The amount of software in modern cars is increasing continuously with traditional electric/electronic (E/E) architectures reaching their limit when deploying complex applications, e.g., regarding bandwidth or computational power. To mitigate this situation, more powerful computing platforms are being employed and applications are developed as distributed applications, e.g., involving microservices. Microservices received widespread adoption and changed the way modern applications are developed.…

CARISMA: CAR-Integrated Service Mesh Architecture
Kevin Klein, Pascal Hirmer, Steffen Becker
https://arxiv.org/abs/2403.04378 https://

CARISMA: CAR-Integrated Service Mesh Architecture
The amount of software in modern cars is increasing continuously with traditional electric/electronic (E/E) architectures reaching their limit when deploying complex applications, e.g., regarding bandwidth or computational power. To mitigate this situation, more powerful computing platforms are being employed and applications are developed as distributed applications, e.g., involving microservices. Microservices received widespread adoption and changed the way modern applications are developed.…

Wireless Distributed Matrix-Vector Multiplication using Over-the-Air Computation and Analog Coding
Jinho Choi
https://arxiv.org/abs/2402.11390 https://

Wireless Distributed Matrix-Vector Multiplication using Over-the-Air Computation and Analog Coding
In this paper, we propose an over-the-air (OTA)-based approach for distributed matrix-vector multiplications in the context of distributed machine learning (DML). Thanks to OTA computation, the column-wise partitioning of a large matrix enables efficient workload distribution among workers (i.e., local computing nodes) based on their computing capabilities. In addition, without requiring additional bandwidth, it allows the system to remain scalable even as the number of workers increases to mit…

[2024-03-07 Thu (UTC), 5 new articles found for cs.DC Distributed, Parallel, and Cluster Computing]

This https://arxiv.org/abs/2310.12615 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csDC_…

Network Abstractions for Characterizing Communication Requirements in Asynchronous Distributed Systems
Whereas distributed computing research has been very successful in exploring the solvability/impossibility border of distributed computing problems like consensus in representative classes of computing models with respect to model parameters like failure bounds, this is not the case for characterizing necessary and sufficient communication requirements. In this paper, we introduce network abstractions as a novel approach for modeling communication requirements in asynchronous distributed system…

This https://arxiv.org/abs/2401.16414 has been replaced.
link: https://scholar.google.com/scholar?q=a

A Causal Model for Quantifying Multipartite Classical and Quantum Correlations
We give an operational definition of information-theoretic resources within a given multipartite classical or quantum correlation. We present our causal model that serves as the source coding side of this correlation and introduce a novel concept of resource rate. We argue that, beyond classical secrecy, additional resources exist that are useful for the security of distributed computing problems, which can be captured by the resource rate. Furthermore, we establish a relationship between resou…

ColonyOS -- A Meta-Operating System for Distributed Computing Across Heterogeneous Platform
Johan Kristiansson
https://arxiv.org/abs/2403.16486 https://

ColonyOS -- A Meta-Operating System for Distributed Computing Across Heterogeneous Platform
This paper presents ColonyOS, an open-source meta-operating system designed to improve integration and utilization of diverse computing platforms, including IoT, edge, cloud, and HPC. Operating as an overlay, ColonyOS can interface with a wide range of computing environments, fostering creation of so-called compute continuums. This makes it possible to develop AI workflows and applications that can operate across platforms. At its core, ColonyOS consists of distributed executors that integrate …

ColonyOS -- A Meta-Operating System for Distributed Computing Across Heterogeneous Platform
Johan Kristiansson
https://arxiv.org/abs/2403.16486 https://

ColonyOS -- A Meta-Operating System for Distributed Computing Across Heterogeneous Platform
This paper presents ColonyOS, an open-source meta-operating system designed to improve integration and utilization of diverse computing platforms, including IoT, edge, cloud, and HPC. Operating as an overlay, ColonyOS can interface with a wide range of computing environments, fostering creation of so-called compute continuums. This makes it possible to develop AI workflows and applications that can operate across platforms. At its core, ColonyOS consists of distributed executors that integrate …

Modeling Distributed Computing Infrastructures for HEP Applications
Maximilian Horzela, Henri Casanova, Manuel Giffels, Artur Gottmann, Robin Hofsaess, G\"unter Quast, Simone Rossi Tisbeni, Achim Streit, Fr\'ed\'eric Suter
https://arxiv.org/abs/2403.14903

Modeling Distributed Computing Infrastructures for HEP Applications
Predicting the performance of various infrastructure design options in complex federated infrastructures with computing sites distributed over a wide area network that support a plethora of users and workflows, such as the Worldwide LHC Computing Grid (WLCG), is not trivial. Due to the complexity and size of these infrastructures, it is not feasible to deploy experimental test-beds at large scales merely for the purpose of comparing and evaluating alternate designs. An alternative is to study t…

[2024-03-06 Wed (UTC), 3 new articles found for cs.DC Distributed, Parallel, and Cluster Computing]

WindGP: Efficient Graph Partitioning on Heterogenous Machines
Li Zeng, Haohan Huang, Binfan Zheng, Kang Yang, Shengcheng Shao, Jinhua Zhou, Jun Xie, Rongqian Zhao, Xin Chen
https://arxiv.org/abs/2403.00331

WindGP: Efficient Graph Partitioning on Heterogenous Machines
Graph Partitioning is widely used in many real-world applications such as fraud detection and social network analysis, in order to enable the distributed graph computing on large graphs. However, existing works fail to balance the computation cost and communication cost on machines with different power (including computing capability, network bandwidth and memory size), as they only consider replication factor and neglect the difference of machines in realistic data centers. In this paper, we p…

[2024-04-05 Fri (UTC), 8 new articles found for cs.DC Distributed, Parallel, and Cluster Computing]

[2024-03-05 Tue (UTC), no new articles found for cs.DC Distributed, Parallel, and Cluster Computing]

This https://arxiv.org/abs/2307.09444 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csDC_…

No distributed quantum advantage for approximate graph coloring
We give an almost complete characterization of the hardness of $c$-coloring $χ$-chromatic graphs with distributed algorithms, for a wide range of models of distributed computing. In particular, we show that these problems do not admit any distributed quantum advantage. To do that: 1) We give a new distributed algorithm that finds a $c$-coloring in $χ$-chromatic graphs in $\tilde{\mathcal{O}}(n^{\frac{1}α})$ rounds, with $α= \bigl\lfloor\frac{c-1}{χ- 1}\bigr\rfloor$. 2) We prove that any di…

Automated Calibration of Parallel and Distributed Computing Simulators: A Case Study
Jesse McDonald, Maximilian Horzela, Fr\'ed\'eric Suter, Henri Casanova
https://arxiv.org/abs/2403.13918

Automated Calibration of Parallel and Distributed Computing Simulators: A Case Study
Many parallel and distributed computing research results are obtained in simulation, using simulators that mimic real-world executions on some target system. Each such simulator is configured by picking values for parameters that define the behavior of the underlying simulation models it implements. The main concern for a simulator is accuracy: simulated behaviors should be as close as possible to those observed in the real-world target system. This requires that values for each of the simulato…

[2024-04-04 Thu (UTC), 5 new articles found for cs.DC Distributed, Parallel, and Cluster Computing]

[2024-03-04 Mon (UTC), 4 new articles found for cs.DC Distributed, Parallel, and Cluster Computing]

[2024-05-03 Fri (UTC), 6 new articles found for cs.DC Distributed, Parallel, and Cluster Computing]

A Communication- and Memory-Aware Model for Load Balancing Tasks
Jonathan Lifflander, Philippe P. Pebay, Nicole L. Slattengren, Pierre L. Pebay, Robert A. Pfeiffer, Joseph D. Kotulski, Sean T. McGovern
https://arxiv.org/abs/2404.16793

A Communication- and Memory-Aware Model for Load Balancing Tasks
While load balancing in distributed-memory computing has been well-studied, we present an innovative approach to this problem: a unified, reduced-order model that combines three key components to describe "work" in a distributed system: computation, communication, and memory. Our model enables an optimizer to explore complex tradeoffs in task placement, such as increased parallelism at the expense of data replication, which increases memory usage. We propose a fully distributed, heuristic-based…

FSD-Inference: Fully Serverless Distributed Inference with Scalable Cloud Communication
Joe Oakley, Hakan Ferhatosmanoglu
https://arxiv.org/abs/2403.15195 …

FSD-Inference: Fully Serverless Distributed Inference with Scalable Cloud Communication
Serverless computing offers attractive scalability, elasticity and cost-effectiveness. However, constraints on memory, CPU and function runtime have hindered its adoption for data-intensive applications and machine learning (ML) workloads. Traditional 'server-ful' platforms enable distributed computation via fast networks and well-established inter-process communication (IPC) mechanisms such as MPI and shared memory. In the absence of such solutions in the serverless domain, parallel computatio…

CASPER: Carbon-Aware Scheduling and Provisioning for Distributed Web Services
Abel Souza, Shruti Jasoria, Basundhara Chakrabarty, Alexander Bridgwater, Axel Lundberg, Filip Skogh, Ahmed Ali-Eldin, David Irwin, Prashant Shenoy
https://arxiv.org/abs/2403.14792

CASPER: Carbon-Aware Scheduling and Provisioning for Distributed Web Services
There has been a significant societal push towards sustainable practices, including in computing. Modern interactive workloads such as geo-distributed web-services exhibit various spatiotemporal and performance flexibility, enabling the possibility to adapt the location, time, and intensity of processing to align with the availability of renewable and low-carbon energy. An example is a web application hosted across multiple cloud regions, each with varying carbon intensity based on their local …

This https://arxiv.org/abs/2305.13385 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csDC_…

A Hybrid Communication Approach for Metadata Exchange in Geo-Distributed Fog Environments
Metadata exchange is crucial for efficient geo-distributed fog computing. Existing solutions for metadata exchange overlook geo-awareness or lack adequate failure tolerance. We propose HFCS, a novel hybrid communication system that combines hierarchical and peer-to-peer elements, along with edge pools. HFCS utilizes a gossip protocol for dynamic metadata exchange. In simulation, we investigate the impact of node density and edge pool size on HFCS performance. We observe a performance improvem…

On Distributed Computation of the Minimum Triangle Edge Transversal
Keren Censor-Hillel, Majd Khoury
https://arxiv.org/abs/2402.13985 https://

On Distributed Computation of the Minimum Triangle Edge Transversal
The distance of a graph from being triangle-free is a fundamental graph parameter, counting the number of edges that need to be removed from a graph in order for it to become triangle-free. Its corresponding computational problem is the classic minimum triangle edge transversal problem, and its normalized value is the baseline for triangle-freeness testing algorithms. While triangle-freeness testing has been successfully studied in the distributed setting, computing the distance itself in a dis…

[2024-05-01 Wed (UTC), 8 new articles found for cs.DC Distributed, Parallel, and Cluster Computing]

[2024-04-01 Mon (UTC), no new articles found for cs.DC Distributed, Parallel, and Cluster Computing]

[2024-03-01 Fri (UTC), 2 new articles found for cs.DC Distributed, Parallel, and Cluster Computing]

[2024-04-30 Tue (UTC), 10 new articles found for cs.DC Distributed, Parallel, and Cluster Computing]

This https://arxiv.org/abs/2003.03255 has been replaced.
link: https://scholar.google.com/scholar?q=a

The Topology of Local Computing in Networks
Modeling distributed computing in a way enabling the use of formal methods is a challenge that has been approached from different angles, among which two techniques emerged at the turn of the century: protocol complexes, and directed algebraic topology. In both cases, the considered computational model generally assumes communication via shared objects, typically a shared memory consisting of a collection of read-write registers. Our paper is concerned with network computing, where the processe…

This https://arxiv.org/abs/2003.03255 has been replaced.
link: https://scholar.google.com/scholar?q=a

The Topology of Local Computing in Networks
Modeling distributed computing in a way enabling the use of formal methods is a challenge that has been approached from different angles, among which two techniques emerged at the turn of the century: protocol complexes, and directed algebraic topology. In both cases, the considered computational model generally assumes communication via shared objects, typically a shared memory consisting of a collection of read-write registers. Our paper is concerned with network computing, where the processe…

[2024-02-29 Thu (UTC), 5 new articles found for cs.DC Distributed, Parallel, and Cluster Computing]

Radical-Cylon: A Heterogeneous Data Pipeline for Scientific Computing
Arup Kumar Sarker, Aymen Alsaadi, Niranda Perera, Mills Staylor, Gregor von Laszewski, Matteo Turilli, Ozgur Ozan Kilic, Mikhail Titov, Andre Merzky, Shantenu Jha, Geoffrey Fox
https://arxiv.org/abs/2403.15721

Radical-Cylon: A Heterogeneous Data Pipeline for Scientific Computing
Managing and preparing complex data for deep learning, a prevalent approach in large-scale data science can be challenging. Data transfer for model training also presents difficulties, impacting scientific fields like genomics, climate modeling, and astronomy. A large-scale solution like Google Pathways with a distributed execution environment for deep learning models exists but is proprietary. Integrating existing open-source, scalable runtime tools and data frameworks on high-performance comp…

Radical-Cylon: A Heterogeneous Data Pipeline for Scientific Computing
Arup Kumar Sarker, Aymen Alsaadi, Niranda Perera, Mills Staylor, Gregor von Laszewski, Matteo Turilli, Ozgur Ozan Kilic, Mikhail Titov, Andre Merzky, Shantenu Jha, Geoffrey Fox
https://arxiv.org/abs/2403.15721

Radical-Cylon: A Heterogeneous Data Pipeline for Scientific Computing
Managing and preparing complex data for deep learning, a prevalent approach in large-scale data science can be challenging. Data transfer for model training also presents difficulties, impacting scientific fields like genomics, climate modeling, and astronomy. A large-scale solution like Google Pathways with a distributed execution environment for deep learning models exists but is proprietary. Integrating existing open-source, scalable runtime tools and data frameworks on high-performance comp…

Lessons Learned from Building Edge Software System Testbeds
Tobias Pfandzelter, David Bermbach
https://arxiv.org/abs/2403.16869 https://

Lessons Learned from Building Edge Software System Testbeds
Edge computing requires the complex software interaction of geo-distributed, heterogeneous components. The growing research and industry interest in edge computing software systems has necessitated exploring ways of testing and evaluating edge software at scale without relying on physical infrastructure. Beyond simulation, virtual testbeds that emulate edge infrastructure can provide a cost-efficient yet realistic environment to evaluate edge software. In this experience paper, we share lesso…

Lessons Learned from Building Edge Software System Testbeds
Tobias Pfandzelter, David Bermbach
https://arxiv.org/abs/2403.16869 https://

Lessons Learned from Building Edge Software System Testbeds
Edge computing requires the complex software interaction of geo-distributed, heterogeneous components. The growing research and industry interest in edge computing software systems has necessitated exploring ways of testing and evaluating edge software at scale without relying on physical infrastructure. Beyond simulation, virtual testbeds that emulate edge infrastructure can provide a cost-efficient yet realistic environment to evaluate edge software. In this experience paper, we share lesso…

This https://arxiv.org/abs/1507.01546 has been replaced.
link: https://scholar.google.com/scholar?q=a

Autonomic Cloud Computing: Research Perspective
As the cloud infrastructure grows, it becomes more challenging to manage resources in such a massive, diverse, and distributed setting, despite the fact that cloud computing provides computational capabilities on-demand. Due to resource variability and unpredictability, resource allocation issues arise in a cloud setting. A Quality of Service (QoS) based autonomic resource management strategy automates resource management, delivering trustworthy, dependable, and cost-effective cloud services th…

Efficient Resource Scheduling for Distributed Infrastructures Using Negotiation Capabilities
Junjie Chu, Prashant Singh, Salman Toor
https://arxiv.org/abs/2402.06938

Efficient Resource Scheduling for Distributed Infrastructures Using Negotiation Capabilities
In the past few decades, the rapid development of information and internet technologies has spawned massive amounts of data and information. The information explosion drives many enterprises or individuals to seek to rent cloud computing infrastructure to put their applications in the cloud. However, the agreements reached between cloud computing providers and clients are often not efficient. Many factors affect the efficiency, such as the idleness of the providers' cloud computing infrastructu…

This https://arxiv.org/abs/2402.17652 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csDC_…

Navigator: A Decentralized Scheduler for Latency-Sensitive ML Workflows
We consider ML query processing in distributed systems where GPU-enabled workers coordinate to execute complex queries: a computing style often seen in applications that interact with users in support of image processing and natural language processing. In such systems, coscheduling of GPU memory management and task placement represents a promising opportunity. We propose Navigator, a novel framework that unifies these functions to reduce job latency while using resources efficiently, placing t…

[2024-04-29 Mon (UTC), 2 new articles found for cs.DC Distributed, Parallel, and Cluster Computing]

[2024-02-28 Wed (UTC), 6 new articles found for cs.DC Distributed, Parallel, and Cluster Computing]

Navigator: A Decentralized Scheduler for Latency-Sensitive ML Workflows
Yuting Yang, Andrea Merlina, Weijia Song, Tiancheng Yuan, Ken Birman, Roman Vitenberg
https://arxiv.org/abs/2402.17652

Navigator: A Decentralized Scheduler for Latency-Sensitive ML Workflows
We consider ML query processing in distributed systems where GPU-enabled workers coordinate to execute complex queries: a computing style often seen in applications that interact with users in support of image processing and natural language processing. In such systems, coscheduling of GPU memory management and task placement represents a promising opportunity. We propose Navigator, a novel framework that unifies these functions to reduce job latency while using resources efficiently, placing t…

Fog enabled distributed training architecture for federated learning
Aditya Kumar, Satish Narayana Srirama
https://arxiv.org/abs/2402.12906 https://…

Fog enabled distributed training architecture for federated learning
The amount of data being produced at every epoch of second is increasing every moment. Various sensors, cameras and smart gadgets produce continuous data throughout its installation. Processing and analyzing raw data at a cloud server faces several challenges such as bandwidth, congestion, latency, privacy and security. Fog computing brings computational resources closer to IoT that addresses some of these issues. These IoT devices have low computational capability, which is insufficient to tra…

[2024-03-29 Fri (UTC), 3 new articles found for cs.DC Distributed, Parallel, and Cluster Computing]

[2024-02-27 Tue (UTC), 3 new articles found for cs.DC Distributed, Parallel, and Cluster Computing]

On the Bit Complexity of Iterated Memory
Guillermo Toyos-Marfurt, Petr Kuznetsov
https://arxiv.org/abs/2402.12484 https://arxiv.org/p…

On the Bit Complexity of Iterated Memory
Computability, in the presence of asynchrony and failures, is one of the central questions in distributed computing. The celebrated asynchronous computability theorem (ACT) charaterizes the computing power of the read-write shared-memory model through the geometric properties of its protocol complex: a combinatorial structure describing the states the model can reach via its finite executions. This characterization assumes that the memory is of unbounded capacity, in particular, it is able to s…

[2024-03-28 Thu (UTC), 7 new articles found for cs.DC Distributed, Parallel, and Cluster Computing]

[2024-02-26 Mon (UTC), 2 new articles found for cs.DC Distributed, Parallel, and Cluster Computing]

Local certification of forbidden subgraphs
Nicolas Bousquet, Linda Cook, Laurent Feuilloley, Th\'eo Pierron, S\'ebastien Zeitoun
https://arxiv.org/abs/2402.12148

Local certification of forbidden subgraphs
Detecting specific structures in a network has been a very active theme of research in distributed computing for at least a decade. In this paper, we start the study of subgraph detection from the perspective of local certification. Remember that a local certification is a distributed mechanism enabling the nodes of a network to check the correctness of the current configuration, thanks to small pieces of information called certificates. Our main question is: For a given graph $H$, what is the …

Uncertainty Estimation in Multi-Agent Distributed Learning for AI-Enabled Edge Devices
Gleb Radchenko, Victoria Andrea Fill
https://arxiv.org/abs/2403.09141

Uncertainty Estimation in Multi-Agent Distributed Learning for AI-Enabled Edge Devices
Initially considered as low-power units with limited autonomous processing, Edge IoT devices have seen a paradigm shift with the introduction of FPGAs and AI accelerators. This advancement has vastly amplified their computational capabilities, emphasizing the practicality of edge AI. Such progress introduces new challenges of optimizing AI tasks for the limitations of energy and network resources typical in Edge computing environments. Our study explores methods that enable distributed data pro…